Low-Rank RNN Adaptation for Context-Aware Language Modeling
نویسندگان
چکیده
منابع مشابه
Low-Rank RNN Adaptation for Context-Aware Language Modeling
A context-aware language model uses location, user and/or domain metadata (context) to adapt its predictions. In neural language models, context information is typically represented as an embedding and it is given to the RNN as an additional input, which has been shown to be useful in many applications. We introduce a more powerful mechanism for using context to adapt an RNN by letting the cont...
متن کاملMLContext: A Context-Modeling Language for Context-Aware Systems
Context awareness refers to systems that can both sense and react based on their environment. The complexity of these systems makes necessary to apply software engineering techniques in their development, such as Model-Driven Software development (MDD). One of the main difficulties that developers of contextaware systems must tackle is how to manage the needed context information. In this paper...
متن کاملA High-rank Rnn Language Model
We formulate language modeling as a matrix factorization problem, and show that the expressiveness of Softmax-based models (including the majority of neural language models) is limited by a Softmax bottleneck. Given that natural language is highly context-dependent, this further implies that in practice Softmax with distributed word embeddings does not have enough capacity to model natural lang...
متن کاملModel Adaptation with Bayesian Hierarchical Modeling for Context-Aware Recommendation
Model adaptation is a process of modifying a model trained with a large amount of training data from the source domain to adapt a speci c similar target domain by using a small amount of adaptation data regarding the target domain. Bayesian hierarchical modeling is well known as a general tool for model adaptation and multi-task learning, and widely used in various areas such as marketing, ecol...
متن کاملLanguage Modeling with Power Low Rank Ensembles
We present power low rank ensembles (PLRE), a flexible framework for n-gram language modeling where ensembles of low rank matrices and tensors are used to obtain smoothed probability estimates of words in context. Our method can be understood as a generalization of ngram modeling to non-integer n, and includes standard techniques such as absolute discounting and Kneser-Ney smoothing as special ...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Transactions of the Association for Computational Linguistics
سال: 2018
ISSN: 2307-387X
DOI: 10.1162/tacl_a_00035